This invention relates to a microscope generating a three-dimensional representation of the observed object, operating on a principle derived from xe2x80x9cimage formation by scattered field inversionxe2x80x9d, tomography and synthetic aperture systems.
2.1. References
[Wolf]: Three-Dimensional Structure Determination of Semi-Transparent Object from Holographic Data, Emil Wolf, Optics Communications, Vol. 1, No. 4, p.153, October 1969.
[Dxc3xa4ndliker]: Reconstruction of the Three-Dimensional-Refractive Index from Scattered Waves, R. Dxc3xa4ndliker, K. Weiss, Optics Communications, Vol. 1, No. p.323, February 1970.
[Fercher]: Image Formation by Inversion of Scattered Field Data: Experiments and Computational Simulation. A. F. Fercher, H. Bartelt, H. Becker, E. Wiltschko, Applied Optics, Vol.18, No 14, p.2427, July 1979
[Kawata]: Optical Microscope Tomography. I. Support Constraint, S. Kawata, O. Nakamnura and S. Minami, Journal of the Optical Society of America A, Vol. 4, No.1, p.292, January 1987
[Noda]: Three-Dimensional Phase-Contrast Imaging by a Computed-Tomography Microscope, Tomoya Noda, Satoshi Kawata and Shigeo Minami, Applied Optics, Vol. 31, No. 5, p.670, Feb. 10, 1992
[Devaney]: The Coherent Optical Tomographic Microscope, A. J. Devaney and A. Schatzberg, SPIE, Vol.1767, p.62, 1992
[Wedberg]: Experimental Simulation of the Quantitative Imaging Properties of Optical Diffraction Tomography, Torolf A. Wedberg and Jacob J. Stamnes, Journal of the Optical Society of America A, Vol. 12, No 3, p.493, March 1995.
[Vishnyakov]: Interferometric Computed-Microtomography of 3D Phase Objects, Gennady N. Vishnyakov and Gennady G. Levin, SPIE Proceedings, Vol.2984, p.64, 1997
[Ausherman]: Developments in Radar Imaging, D. A. Ausherman, A. Kozma, J. L. Walker, H. M. Jones, E. C. Poggio, IEEE Transactions on Aerospace and Electronic Systems, Vol. 20, No. 4, p.363, July 1984.
[Goodman]: Synthetic Aperture Optics, Progress in Optics, Vol. VIII, 1970, North Holland Publishing Company.
[Walker]: Range-Doppler Imaging of Rotating Objects, Jack L. Walker, IEEE transactions on Aerospace and Electronic Systems, Vol. 16, No. 1, p.23, Januasy 1980.
[Brown]: Walker model for Radar Sensing of Rigid Target Fields, William M. Brown, IEEE Transactions on Aerospace and Electronic Systems, Vol. 16, No 1, p.104, January 1980.
[Turpin 1]: U.S. Pat. No. 5,384,573
[Turpin 2]: Theory of the Synthetic Aperture Microscope, Terry Turpin, Leslie Gesell, Jeffrey Lapides, Craig Price, SPIE Proceedings, Vol. 2566, p.230, 1995
[Turpin 3]: The Synthetic Aperture Microscope, Experimental results, P. Woodford, T. Turpin, M. Rubin, J. Lapides, C. Price, SPIE Proceedings, Vol. 2751 p.230, 1996
[Lauer 1]: Patent WO 98/13715
2.2. Description of the Prior Art
A three-dimensional object may be characterized optically by a certain number of local parameters, for example its index and its absorptivity at each point. Mathematically, this may be expressed by the data at each point of a complex number which is a function of the local parameters at the considered point. A three-dimensional spatial representation of the object can then be expressed in the form of a three-dimensional array of complex numbers.
By carrying out the three-dimensional Fourier transform of this three-dimensional spatial representation, a three-dimensional frequency representation of the object is obtained.
[Wolf] showed that a three-dimensional representation of a weakly diffracting object can be obtained from the acquisition of the wave diffracted by this object when it is illuminated successively by a series of plane waves of variable direction. [Wolf] also determined the maximum resolution thus obtainable, expressed as a function of the illuminating wavelength. This resolution corresponds to a maximum period of xcex/2 for the sinusoidal components the object""s representation, i.e. a sampling period in the Nyquist sense of xcex/4 which is a resolution twice as fine as that of conventional microscopes. [Dxc3xa4ndliker] improved [Wolf]""s formalism and provided a geometrical interpretation of it. From the wave diffracted by the object under a given illumination, part of the three-dimensional frequency representation of the object is obtained. This part is a sphere in a three-dimensional frequency space. By combining the spheres thus obtained for various illuminating waves, the frequency space can be filled, obtaining the three-dimensional frequency representation of the object. The latter can then be inverse transformed to obtain a spatial representation.
[Fercher] designed a microscope constituting the first practical application of the principles defined by [Wolf] and [Dandliker]. In that microscope, the wave diffracted by the object is picked up on a receiving surface on which it interferes with a reference wave not having passed through the object and the phase of which can be modified. From several interference figures differing from each other by the phase of the reference wave, [Fercher] obtains, at each point of the receiving surface, the amplitude and the phase of the wave diffracted by the object.
[Fercher] does not use several successive illumination waves but several illuminating waves generated simultaneously by means of a diffraction grating, thus limiting the number of possible illumination directions, even though the use of several successive illuminating waves does not present any particular technical difficulty. The reason for this choice is not clearly explained. However, it appears that this technique is adopted in order to obtain illuminating waves all having the same phase at a given point of the image. In fact, the equation (1) of the document [Wolf] assumes that each illuminating wave has a zero phase at the point of origin of the position vectors.
The method defined by [Wolf], [Dxc3xa4ndliker] and [Fercher] is generally called xe2x80x9cimage formation by scattered field inversionxe2x80x9d. Another conventional approach for obtaining three-dimensional images is tomography. Tomography, used for example in x-ray techniques, consists in reconstructing an image from a set of projections of this image along different directions. Each projection depends linearly upon a three-dimensional density function characterizing the object. From a sufficient number of projections it is possible to reconstitute the object by reversing this linear correspondence.
Tomography was adapted to optical microscopy by [Kawata]. In his tomographic microscope, a plane and non-coherent illuminating wave of variable direction is used. This illuminating wave passes through a sample and then a microscope objective focussed in the plane of the sample. It is received on a receiving surface placed in the plane in which the objective forms the image of the sample. Because the illumination is non-coherent, the intensities coming from each point of the object are added and the image intensity produced on the receiving surface consequently depends linearly on the three-dimensional density function characterizing the absorptivity of the object. From a sufficient number of images it is possible to reconstitute the image by reversing this linear correspondence. This microscope differs from usual tomographic systems in that the linear correspondence between the density function of the object and a given image is not a projection, but is characterized by a three-dimensional optical transfer function.
This microscope is not very suitable for obtaining images that take into account the index of the sample. [Noda] designed a modified microscope enabling this phase to be taken into account. The initial idea of that microscope [Noda] is to use phase contrast for obtaining an image which depends on the index of the sample, and to adapt to this configuration the linear correspondence reversal principle already implemented by [Kawata]. The use of the [Noda] microscope is however limited to the study of non-absorbing objects whose index variations are extremely small.
[Noda]""s text does not refer to holography nor to xe2x80x9cimage formation by scattered field inversion,xe2x80x9d but the operating principle involved can be interpreted within this framework. In fact, the technique adopted by [Noda] is tantamount to using on the receiving surface a reference wave consisting of the illuminating wave alone. From the images received for a set of illuminating waves of variable direction, a three-dimensional frequency representation is obtained. The complex wave detected on the receiving surface is replaced here by a pure imaginary value obtained by multiplying by j the real value obtained using the single reference wave constituted by the illuminating wave shifted xcfx80/2 in phase. If the reference wave is sufficiently higher, at each point of the receiving surface, than the diffracted wave, then the quantity thus obtained is the imaginary part of the complex wave really received on the receiving surface, the phase reference being the phase of the illuminating wave. The object generating a pure imaginary wave equivalent to that detected by [Noda] on the receiving surface is made up of the superposition of the observed real object and a virtual object whose complex spatial representation is obtained from that of the real object by symmetry in relation to the plane of the object corresponding to the receiving surface, and by reversing the sign of the real part. Using the imaginary part thus detected in a manner similar to that used by [Fercher] for the detected complex wave, a function representing the superposition of the real object and the virtual object is generated in frequency representation. During each acquisition, the two-dimensional frequency representation obtained by performing the Fourier transform of the value detected on the receiving surface includes a part corresponding to the real object and a part corresponding to the virtual object, which coincide only at the point corresponding to the illumination frequency. It is thus possible to select only the part corresponding to the real object, so as to obtain a representation thereof. [Noda] in fact uses the superposition of the real object with the virtual object which he symmetrizes in relation to the plane of the object corresponding to the receiving surface, thus obtaining a pure imaginary representation corresponding to the imaginary part of the representation that would be obtained using [Wolf]""s method.
The theoretical explanations given in the document [Noda] are very different from those presented here and are perfectly valid. The principle consisting in reversing a filter by multiplication in the frequency domain, as applied by [Noda], is found to be equivalent to the explanations given above, although obtained by different reasoning. It may be considered that FIGS. 2 and 3 of the document [Noda] illustrate how the three-dimensional frequency representation of the object is generated from two-dimensional frequency representations.
[Devaney] proposed a tomographic microscope whose operating mode is derived essentially from the method of [Wolf]. In the [Devaney] microscope the reference wave coincides with the illuminating wave. Consequently, that microscope does not have means for varying the phase of the reference wave. As in the case of [Noda], the detected wave consequently corresponds to that which would be formed by the superposition of a real object and a virtual object. [Devaney] solves the problem by placing the receiving surface outside of the object, so that the real object and the virtual object do not overlap. When the direction of the illuminating wave varies, only one of the two objects is reconstituted. Two variants of the microscope are presented: a first in which the object is fixed and the direction of the illuminating wave is variable, and a second in which the object rotates around a fixed point, the illuminating wave then having a fixed direction in relation to the receiver. The first version of the [Devaney] microscope was built by [Wedberg].
Another approach allowing the adaptation of tomography to the production of phase images is that of [Vishnyakov]. [Vishnyakov] introduces a reference wave distinct from the illuminating wave and carries out a detection of the wave received on a receiving surface in accordance with a method similar to the one used by [Fercher]. He then generates a profile characteristic of the phase difference between the received wave and the illuminating wave. As this phase difference is considered to be the projection of the index along the direction of the illuminating wave, it regenerates the distribution of the index in the object according to the tomographic method used conventionally in x-ray techniques. This method may be compared to a method of the [Wolf] type, but in which the sphere portion acquired in the frequency space would be assimilated with a plane portion, which is largely unjustified in the case of an objective with a large aperture such as the one used here.
The technique of the synthetic aperture radar is an imagery method used in the field of radar waves and which was considered very early for application to the field of optics. [Ausherman] presents the history of this technique. The application of the synthetic aperture radar technique to the field of optical wavelengths would make it possible in principle to obtain images of an observed object. However, in order for the technique to be feasible, it is necessary, at all times, to have the position values, within a reference coordinate system linked to the object, of each element of the transmitter-receiver assembly. These values must be known to within a fraction of a wavelength. This is achievable in the area of radar frequencies, in which wavelengths are macroscopic and can be, for example, of a few tens of centimeters. In the optics field, where wavelengths are sub-micrometric, this is difficult to achieve. This problem is the basic reason for which the system is difficult to adapt to optics, as indicated in [Goodman], Pages 36 to 39.
[Walker] and [Brown] formalized the synthetic aperture radar method in a form similar to that already obtained by [Wolf] for optical systems. This formalism was originally used by [Walker] with a radar imagery method in which the transmitter-receiver assembly is fixed and in which the object rotates around a fixed point. This overcomes the problem of determining the position of the object at each instant.
[Turpin] recently described several microscopes constituting an adaptation of synthetic aperture radar principles to the field of optics.
In the microscope implemented by [Turpin 3], the material configuration used complies with the principle used in [Walker] to overcome the problem consisting in determining the position of the object at each instant, i.e. the transmitter and receiver are fixed and the object rotates around a fixed axis. This microscope is also similar to the second version of the [Devaney] microscope. Since the rotation axis, the transmitter and the receiver are fixed, the position of the transmitter-receiver assembly within a reference coordinate system linked to the object can be known with the required accuracy.
However, effective resolution calls not only for a mechanical system making it possible to determine the movement of the object, but also the taking of this movement into account in the definition of algorithms and/or appropriate adjustment of the system. In the absence of special precautions, the point of origin of the reference wave moves in relation to the object over a circle centered on the rotation axis of the object. If this movement is significant, this effect destroys the image. If this movement is small, resolution in the plane of this circle is affected in proportion to the amplitude of the movement.
To solve this problem in the absence of any specific compensation algorithm, the point of origin of the reference wave should be on the rotation axis of the object. This condition is in principle difficult to obtain. [Turpin] does not mention this problem and does not specify any means of appropriate adjustment.
This problem can however be solved when use is made, for example, of a flat specimen, by performing an adjustment intended to comply with the following conditions:
(i)xe2x80x94The image of the reference wave on the xe2x80x9cimage CCD arrayxe2x80x9d of FIG. 1 in document [Turpin 3] must be a point image.
(ii)xe2x80x94When the object makes a rotation of 180 degrees, the image obtained must be symmetrized in relation to an axis passing through the point image of the reference wave.
The position of the xe2x80x9cimage CCD arrayxe2x80x9d must be adjusted to comply with (i).
The position of the entire receiver must be adjusted to comply with (ii).
This solution is however not perfect, depending essentially on a visual interpretation. It can be used reasonably only for very simple objects.
The microscope described in [Turpin 3] is a particular case of the generalized system described in [Turpin 1] and [Turpin 2]. The generalized system specifies that the illuminating wave and/or the position of the receiver can vary. However, the material configurations proposed do not allow the solution of the problem consisting in determining, to within a fraction of a wavelength, the position of the transmitter and the receiver in relation to the object. In fact, the illuminating wave of variable direction is produced by mechanical devices which cannot be controlled with sub-micrometric accuracy.
The microscope described in [Lauer 1] allows the generation of the frequency representation of a wave coming from the object and the reconstitution of the object from several of these representations. The method used in [Lauer 1] is not directly related to the one described by [Wolf]. In fact, in the case in which it recombines several frequency representations to obtain the representation of the object:
it uses spatially incoherent lighting and not plane illuminating waves
it combines the frequency representations of the waves received by intensity summing in the spatial domain.
The image obtained by [Lauer 1] is affected by a residual granularity effect, does not allow a differentiation of the refractive index and of absorptivity, and does not yield the theoretical accuracy indicated by [Wolf].
3.1. Problem to Be Solved by the Invention
Systems based on xe2x80x9cimage formation by scattered field inversion,xe2x80x9d xe2x80x9ctomographicxe2x80x9d systems or xe2x80x9csynthetic aperturexe2x80x9d systems appear to be equivalent to each other, at least when examined from only the coherent optical domain. Two classes of systems may be distinguished depending on the reference wave generation mode:
the microscopes of [Noda] and [Devaney] use a reference wave coinciding with the illuminating wave.
the microscopes of [Fercher], [Vishnyakov] and [Turpin] use a reference wave distinct from the illuminating wave.
Microscopes of the first category exhibit limitations in terms of observed image size, characteristics imposed on the object, or displayable parameters. These limitations are due to the fact that, in the absence of a reference wave distinct from the illuminating wave, it is not possible to acquire under good conditions the complex value of the wave received on the receiving surface. It is thus necessary to use various expedients to eliminate spurious images and various disturbances generated by the drawbacks of the acquisition method.
Microscopes of the second group allow this problem to be solved. In principle, microscopes of the second group should allow the representation of the refractive index and the absorptivity of the object in three dimensions with quarter-wavelength accuracy, according to the theory devised by [Wolf] and [Dxc3xa4ndliker]. Such performance is clearly higher than that of all existing optical microscopes including the confocal microscope, and these microscopes should in all logic have led to industrial applications. However, none of these microscopes has as yet made it possible to obtain quality images comparable to those produced, for example, by the [Noda] microscope, and thus these microscopes have not gone beyond the experimental stage. The reason for which these microscopes have never yielded high quality images, despite their theoretical possibilities, has never been clearly determined.
A first approach to the problem is contained implicitly in the equation (1) of the document [Wolf]: all the illuminating waves must have the same phase at the origin of the representation. However, in an optical system, the only values accessible to measurement are the phase differences between a reference wave and a wave to be analyzed. The fact that at the illuminating waves all have the same phase at a given point of the object, as indicated implicitly by the equation (1) in the document [Wolf], is thus not sufficient to ensure the proper functioning of the system: it is also necessary for the reference wave to meet appropriate conditions, so that the phase differences accessible to measurement lead to correct results.
A second approach to the problem is provided by [Goodman], Pages 36 to 39, in the terms of the numerical aperture radars: the position of the transmitter and receiver must be determined with an accuracy within the wavelength, and this is not feasible in the optical domain.
If the illuminating wave has a variable direction, these two approaches are similar: in fact, an indetermination on the position of the transmitter results, among other effects, in a phase shift of the illuminating wave at the origin of the representation. We shall confine ourselves here to these systems, i.e. to the microscopes of [ercher] and [Vishryakov], and to the versions of the microscope of [Turpin] which include a variable direction illuminating wave.
For example, in the microscopes of [Turpin], the illuminating wave is generated by a mechanical device. This device does not allow the illuminating wave phase to be controlled. When two successive three-dimensional images are taken, a given illuminating wave characterized by its direction will not have the same phase difference with the reference wave in each case. The wave detected on the receiving surface will consequently not have the same phase either, and hence finally that two successive three-dimensional images taken will not lead to the same result, even in the absence of any noise. This example highlights the basic problem which has up to now limited the performance of microscopes of the second group: lack of control of the phase difference between the illuminating wave and the reference wave leads to non-reproducibility of the results obtained and, in general, results which do not correspond to what is expected considering the theoretical approach of [Wolf].
3.2. Solution of Problem According to the Invention
The entire three-dimensional frequency representation of an object can be multiplied by a complex number Aejxcex1. It will then be said that the frequency representation of the object is affected by a phase shift xcex1 and by a multiplicative factor A. If an object is characterized by its three-dimensional frequency representation, its spatial representation can be obtained by taking the inverse Fourier transform of this frequency representation. If the frequency representation of the object is affected by a phase shift xcex1 and a multiplicative factor A, its spatial representation is affected by the same phase shift and the same multiplicative factor, corresponding to a modification of the function giving the complex number associated with a point as a function of the local parameters at this point.
It is also possible to multiply each point of the three-dimensional representation by a complex number Aejxcex1 depending on the point. Two different points of the three-dimensional frequency representation can then be affected by a phase shift and a multiplicative factor which are different. By performing the inverse Fourier transform of a frequency representation in which the phase shift and/or the multiplicative factor depend on the considered point, a modified representation of the object is obtained in which the complex number associated with a point depends not only on the local parameters at the considered point, but also on the local parameters at a set of other points. This modified representation of the object is a filtered spatial representation, the filter having a frequency representation consisting of the numbers Aejxcex1 defined at each point. Depending on the characteristics of this filter, more or less correct frequency representations will be obtained.
In the microscopes of [Turpin], the illuminating wave is generated by a mechanical device. With each change of direction of the illuminating wave, a random phase shift of this wave occurs, and hence a random phase shift in the corresponding part of the three-dimensional frequency representation.
In the document of [Vishnyakov], and for the same reasons, the phase of the illuminating wave varies randomly with each change of illumination direction. Equation (2) on Page 67 of the document [Vishnyakov] should be replaced by "psgr"(x,y)="PHgr"(x,y)+x sin xcex1+xcfx86, where xcfx86 is the phase of the illuminating wave at the origin of the three-dimensional representation obtained. When the direction of the illuminating wave varies, the value of xcfx86 varies. Non-determination of the correct value of xcfx86 results in the addition of a constant to each projection obtained, this constant varying randomly between two projections. This consequently makes inexact the assimilation of the phase profile obtained with the projection of the index. The method of [Vishnyakov] is roughly equivalent to the method of [Wolf] in which a portion of the sphere of the frequency space, obtained from a given illuminating wave, would have been assimilated to a plane. The non-determination of xcfx86 is equivalent to a random phase shift of the entire part of the two-dimensional frequency representation generated from a given illuminating wave. However, other sources of error are added to this effect, and in particular the fact that a conventional tomographic reconstruction is used.
In the document of [Fercher], owing to the use of a diffraction grating to generate the three illuminating waves simultaneously, there is no random phase shift in the illuminating waves. However, a detailed analysis of the system shows that, to obtain the same phase shift in the parts of the frequency representation obtained from each illuminating wave, the virtual image, in the object, of the point of focus of the reference wave, must coincide with a point in which the illuminating waves all have the same phase. This calls for very precise adjustment of the position of the reference wave origin. As the document of [Fercher] does not contain any mention of such an adjustment, it is likely that it has not been carried out. In any case, the solution adopted by [Fercher] limits significantly the number of illuminating waves that may be used and the aperture under which the wave coming from the object can be acquired.
In existing microscopes using variable direction illuminating waves and a reference wave distinct from the illuminating wave, the parts of the frequency representation obtained from different illuminating waves are thus affected by different phase shifts. Consequently, the inversion of the frequency representation obtained generates a filtered representation which is in general of rather poor quality and which is moreover not reproducible owing to the random nature of the phase shift.
The invention consists in providing a microscope in which the direction of the illuminating wave is variable, but comprising means for generating a three-dimensional representation of the object in which the distribution of the phase shift affecting each point obtained, in frequency representation, is concentrated around a constant value. Ideally, this phase shift should be constant, but the existence of disturbances such as Gaussian noise, a residual spherical aberration, or a small imprecision in controlling the phase difference between the reference wave and the illuminating wave, creates a certain spread of the distribution around the constant value.
In existing microscopes using variable direction illuminating waves and a reference wave distinct from the illuminating wave, and in the case in which a large number of distinct illuminating waves are used, such as in [Turpin], this phase shift tends to be of a random nature and its distribution is hence roughly homogeneous over the interval [0,2xcfx80] . Where a limited number of illuminating waves are used, as in [Fercher], the phase shift distribution exhibits peaks of comparable level centered on several distinct values corresponding to the phase shifts affecting the sub-representations obtained from each illuminating wave.
The fact that the phase shift affecting each point, in frequency representation, is roughly constant, constitutes a new functionality of the microscope which allows, for example, a better quality spatial representation. However, the generation of a three-dimensional representation of the observed object does not necessarily constitute the final purpose sought by the microscope user. For example, the microscope may be used to read three-dimensional optical memories. In this case, the data can be encoded before being stored in the optical memory. The microscope then allows a three-dimensional representation of this optical memory to be obtained from which the data can be decoded. The three-dimensional representation of the object is then a calculation intermediary finally enabling the decoded data to be obtained.
3.3. Vocabulary Used and General Considerations
A three-dimensional object can be characterized optically by a certain number of local parameters. These parameters can, for example, be its refractive index and its absorptivity at each point, or its absorptivity and one of its indices in the case of a non-isotropic material. It is possible to define at each point a complex number which is a function of certain of these local parameters, this function generally being defined uniquely in the entire spatial representation and hence not depending on the considered point. It will then be possible to express a three-dimensional spatial representation of the object in the form of a three-dimensional array of complex numbers. The dependency between the complex number and the local parameters can be defined in various ways. For example, this complex number may reduce to a real number characterizing the index, as in the microscope of [Noda]. The definition that will be used most often will however be of the type given by [Wolf], but with a xe2x80x9ccomplex indexxe2x80x9d representing both the refractive index and absorptivity. For example, the real part of the complex number may be proportional to absorptivity and its imaginary part to the refractive index. A complex number may be used which is obtained by rotating the preceding in the complex plane, corresponding to a phase shift. In every case, the three-dimensional spatial representation of the object is unique if the correspondence between the complex number and the local parameters has been defined and the central part of the representation has also been defined.
Performing the three-dimensional Fourier transform of this three-dimensional spatial representation, a three-dimensional frequency representation of the object is obtained.
The phase shift affecting a point of the three-dimensional frequency representation of the object is defined when the representation of the object has been defined uniquely, i.e. when the complex function of the local parameters and the point of origin characterizing the spatial representation have been defined.
When these parameters are not specified, it may be considered that the phase shift is defined in relation to the spatial representation which coincides at best with the obtained three-dimensional representation of the object.
The term xe2x80x9cthree-dimensional representation of an objectxe2x80x9d will designate all the numerical data characterizing the spatial or frequency representation of the object, independent of the manner in which these data are combined or stored, for example in a computer memory. This representation may be expressed, for example:
in the spatial domain, in the form of a complex number dependent on spatial coordinates
in the frequency domain, in the form of a complex number dependent on the spatial frequency
in any other manner when the xe2x80x9cthree-dimensional representation of the objectxe2x80x9d allows the spatial or frequency representation of the object to be obtained by a known algorithm.
Part of the three-dimensional frequency representation of the object will be called frequency sub-representation of the object, and the term sub-representation will designate all the corresponding data, independent of how they are combined or stored.
A wave reaching a receiving surface is entirely characterized by its amplitude and its phase along each polarization direction and at every point of the receiving surface. A scalar representation of the wave may also be adopted by limiting oneself, for example, to a single polarization direction, the wave then being characterized by a single phase and a single intensity at every point of the receiving surface. From the wave measured on a receiving surface, a frequency sub-representation of the observed object can be generated. This sub-representation is two-dimensional in that it constitutes a sphere portion in the space of the frequencies, as indicated by [Dxc3xa4ndliker]. A calculation intermediary may be constituted by a frequency representation of the wave, defined by the wave phase and intensity on each wave vector, a scalar representation being adopted.
The frequency representation of the wave is two-dimensional and can be projected on a plane without any loss of information. Such a projection yields a plane image that will be called xe2x80x9cfrequency plane imagexe2x80x9d. In a system such as that of [Turpin] or that of [Fercher], such a frequency plane image is obtained directly on the receiving surface. In other systems, such as the second embodiment of the present invention, such a frequency plane image is obtained by two-dimensional Fourier transformation of the scalar representation obtained directly on the receiving surface. A modified frequency plane image can also be obtained from several frequency plane images differing from each other in the polarization of the illuminating wave and the direction of analysis of the wave received on the receiving surface. The frequency plane image can constitute a calculation intermediary making it possible to obtain the frequency representation of the wave and then a corresponding sub-representation of the object.
The term xe2x80x9ctwo-dimensional frequency representationxe2x80x9d will designate either a two-dimensional frequency representation or a two-dimensional part of a three-dimensional frequency representation. In particular, it can designate either a frequency plane image, a frequency representation of a wave, or a two-dimensional sub-representation of the object.
The term xe2x80x9clensxe2x80x9d will designate, throughout the text, either simple lenses, compound lenses, or achromats, generally designed to limit optical aberration.
In the rest of the text, five embodiments are described, referred to as Embodiments 1, 2, 3, 4 and 5.
3.4. Obtaining the Three-Dimensional Frequency Representation Directly
For each direction of the illuminating wave, a frequency sub-representation of the object is obtained by direct application of the methods defined in [Fercher] and [Turpin]. In the systems of [Fercher] and [Turpin], the microscope is built so that the different sub-representations obtained are affected by different phase shifts. According to one embodiment of the invention, the microscope is built so that these phase shifts are constant. This variant of the invention implies:
(i)xe2x80x94that the microscope is built so that the phase difference between an illuminating wave and the reference wave with which it interferes is reproducible. This condition implies an appropriate construction of the microscope. This is shown, in the absence of vibrations, in Embodiments 3, 4 and 5. Embodiments 1 and 2 are affected by the same problems as the microscope of [Turpin]: the phase difference between reference wave and illuminating wave varies randomly owing to the overall mechanical design. Embodiments 1 and 2 consequently do not allow compliance with this first condition. Embodiments 3 to 5 enable this condition to be complied with because of the different design of the illuminating and reference wave generating system.
(ii)xe2x80x94that the microscope is built so that, through appropriate adjustment, the phase difference between the illuminating wave and the reference wave can be made constant. Embodiment 3 does not allow this condition to be met because there is no particular point at which all the illuminating waves have the same phase, which would be necessary in order to meet this condition since the reference wave used is spherical and constant. Embodiment 4, quite similar moreover to Embodiment 3, allows this condition to be met because, with an appropriate control of the beam deflection system, it is possible to generate illuminating waves whose phase at a given point is constant. Embodiment 5 also meets this condition.
(iii)xe2x80x94that the position of the optical elements is adjusted appropriately so that the phase difference between the illuminating wave and the reference wave is in fact constant. This adjustment is described in 8.6. for Embodiment 4 and in 9.20. for Embodiment 5.
Conditions (ii) and (iii) imply that the phase difference between the illuminating wave and the reference wave is constant. It is possible to define a virtual wave present in the object and such that its image, through the optical device modifying the wave coming from the object between the object and the receiving surface, is the reference wave. The phase difference between the illuminating wave and the reference wave means here the phase difference between the illuminating wave and the component of this virtual wave on the wave vector of the illuminating wave.
3.5. Phase Correction Method
When the microscope is not built so that the phase shift affecting each frequency sub-representation is constant, the differences between the phase shifts affecting each sub-representation must be determined and compensated if necessary.
3.5.1. General Phase Correction Method
Part of the three-dimensional representation of the object is considered:
consisting of a subset A of the three-dimensional representation.
characterized by a function a(f) defined on A and conventionally zero outside of A, where f is the spatial frequency vector and a(f) the value of the representation on this spatial frequency.
affected by a Gaussian noise, the standard deviation of the noise on a given frequency f being "sgr"a(f).
This sub-representation will be designated by the expression xe2x80x9csub-representation RAxe2x80x9d. It will be said that A is the support of RA.
A second part of the three-dimensional representation of the object is considered, consisting of a subset B of the three-dimensional representation, characterized by a function b(f) defined on B, affected by a Gaussian noise "sgr"b(f), designated by the expression xe2x80x9csub-representation RB.xe2x80x9d
These two parts of the representation are phase shifted in relation to each other and are assumed to have a non-zero intersection. From these two parts will be generated a sub-representation RC defined on a set C=A∪B (C is the union of A and B) and defined by a function c(f) affected by a Gaussian noise "sgr"c(f).
It is possible to proceed in two steps:
a complex ratio between the two representations can be obtained, for example, with the formula:   r  =                    ∑                  f          ∈          E                    ⁢              xe2x80x83            ⁢                        a          ⁢                      xe2x80x83                    ⁢                      (            f            )                    ⁢                      xe2x80x83                    ⁢                                    b              ⁢                              xe2x80x83                            ⁢                              (                f                )                                      _                                                              σ              a              2                        ⁢                          xe2x80x83                        ⁢                          (              f              )                                +                                    σ              b              2                        ⁢                          xe2x80x83                        ⁢                          (              f              )                                                          ∑                  f          ∈          E                    ⁢                                    "LeftBracketingBar"                          b              ⁢                              xe2x80x83                            ⁢                              (                f                )                                      "RightBracketingBar"                    2                                                    σ              a              2                        ⁢                          xe2x80x83                        ⁢                          (              f              )                                +                                    σ              b              2                        ⁢                          xe2x80x83                        ⁢                          (              f              )                                          
xe2x80x83where the sums are over a set E of frequency vectors included in the intersection of the two sets A and B, or E⊂(A∩B) , the set E being if necessary limited to points for which the signal-to-noise ratio is sufficiently high. The phase difference between the two representations is the argument of r. The phase difference thus calculated is a good approximation of the most probable phase difference knowing the values of the representations RA and RB over the set E.
the representation RB may be xe2x80x9cphase-correctedxe2x80x9d in relation to RA by multiplying it by the ratio r:
b(f )←r.b(f) where the sign←means that b(f) is replaced by r.b(f).
the function c can be obtained, for example, by the formula:       c    ⁢          xe2x80x83        ⁢          (      f      )        =                              a          ⁡                      (            f            )                                                σ            a            2                    ⁢                      xe2x80x83                    ⁢                      (            f            )                              +                        b          ⁡                      (            f            )                                                σ            b            2                    ⁢                      xe2x80x83                    ⁢                      (            f            )                                              1                              σ            a            2                    ⁢                      xe2x80x83                    ⁢                      (            f            )                              +              1                              σ            b            2                    ⁢                      xe2x80x83                    ⁢                      (            f            )                              
xe2x80x83The values thus assigned to the representation RC are the most probable values, knowing the phase-corrected representations RA and RB.
the function "sgr"c(f) can be obtained, for example, by the formula:       1                  σ        c        2            ⁢              xe2x80x83            ⁢              (        f        )              =            1                        σ          a          2                ⁢                  xe2x80x83                ⁢                  (          f          )                      +          1                        σ          b          2                ⁢                  xe2x80x83                ⁢                  (          f          )                    
Both of the preceding operations constitute the grouping of RA and RB.
More detailed explanations on the calculation of these functions in the form of arrays are given in Paragraph 7.17.1. The formulas indicated above for phase correction carry out an intensity normalization simultaneously, which however is not indispensable.
This method makes it possible, from the two sub-representations RA and RB whose supports A and B have a non-zero intersection, to obtain a sub-representation RC corresponding to the superposition of RA and RB.
If the three-dimensional frequency representation of the object must be reconstituted from many sub-representations whose phases are not known, the above method, applied iteratively, allows the grouping of all these sub-representations. For example, it is possible to begin with a given sub-representation, and group it with a second sub-representation. Then, it is possible to begin with the sub-representation generated by this grouping, and group it with a further sub-representation. By repeating this grouping operation until all the sub-representations have been integrated in a overall representation, one finally obtains the three-dimensional frequency representation of the object. The only conditions to be met in order for this method to succeed are:
that no sub-representation or group of sub-representations has an empty intersection with all the other sub-representations.
that the object does not have an excessively singular frequency representation, which would be for example zero on a set of points separating in two its three-dimensional frequency representation.
These conditions are easily met for all biological objects as soon as a large number of representations is acquired.
For example:
an embodiment may be derived from the system of [Fercher] in which the three simultaneous illuminating waves produced by the diffraction grating have been replaced by a single illuminating wave of variable direction. In this case, a sub-representation is constituted by a two-dimensional frequency representation obtained for a given illuminating wave.
an embodiment can be derived from the microscope implemented in [Turpin 3] in which, in addition to the rotation of the object, variations in the direction of the illuminating wave have been authorized. In this case, a sub-representation is constituted by all the two-dimensional frequency representations obtained for a given illuminating wave when the object is rotated.
in the case of Embodiment 5, Paragraph 9.19., a sub-representation is constituted by a two-dimensional frequency representation obtained for a given illuminating wave. A small number of sub-representations is first grouped in a basic representation having a non-zero intersection with all the other sub-representations obtained under similar conditions. All the representations are then phase-corrected in relation to the basic representation, and then an overall representation is generated.
in Embodiments 3, 4 and 5, four intermediate sub-representations are generated each time, as explained in 7.17.1.1. These four representations are grouped in a single representation by the application of this general method.
According to a variant of the invention, the microscope thus comprises means for:
determining, for each sub-representation RB, a coefficient characterizing the phase difference between this sub-representation and another sub-representation, part of a sub-representation or group of sub-representations RA, this coefficient being calculated from values of RA and RB over a set included in the intersection of the supports of RA and RB.
correct the phase of RB so as to obtain for RB the same phase reference as for RA.
The simplest way to correct the phase of RB is to carry out a multiplication by the coefficient r as indicated above. However, this correction can also be carried out by physical means, in which case the phase does not need to be corrected during the calculation stage. An example of such an embodiment is described in 7.18.7.
The phase difference affecting each sub-representation can be recalculated with each acquisition. This is necessary in Embodiments 1 and 2, for which these differences are not reproducible. In the case of Embodiments 3, 4 and 5, this phase difference is reproducible and can consequently be measured during a phase preparatory to acquisition. An example of such an embodiment is described in 7.18.1.
According to a variant of the invention, the microscope comprises means for:
determining, for a given sub-representation RB, the frequency representation RC resulting from the grouping of RB with another sub-representation RA.
determining a coefficient characteristic of the noise affecting RC defined over the entire support of RC, obtained from a coefficient characteristic of the noise affecting RA and defined on the support of RA, and from a coefficient characteristic of the noise affecting RB and defined on the support of RB.
The methods used can differ from the formalism set forth above. For example, in Embodiment 1, the quantity   1            σ      c      2        ⁢          xe2x80x83        ⁢          (      f      )      
is assimilated with the number N of frequency representations reaching a given point.
The calculations can be grouped: after phase correction of each sub-representation, they can be grouped in an overall representation, without calculating each intermediate sub-representation. This is what is done in all the embodiments to group two-dimensional sub-representations into complete or partial three-dimensional sub-representations.
Representations of the object can be calculated without passing formally through its three-dimensional frequency representation. For example, in 7.17.3.3., a confocal representation of the object is generated using, for the final frequency representation, a value at each point which is the sum of the values obtained for each representation reaching this point. The representation thus obtained is not strictly speaking a frequency representation of the object, but nevertheless carries information on this object. It is also possible to generate real representations of the refractive index or absorptivity. These representations can be generated simply through the frequency representation of the object, but it is also possible to modify the algorithms so as not to formally use this procedure.
3.5.2. Absolute Phase Correction
The method set forth in 3.5.1. makes it possible to obtain a three-dimensional representation of the object. However, the overall phase of this three-dimensional representation remains arbitrary.
In the three-dimensional spatial representation of the object, obtained from the three-dimensional frequency representation by taking the inverse Fourier transform, the complex number associated with each point characterizes the absorptivity and the refractive index of the considered point. If the overall phase of the three-dimensional representation is chosen appropriately, the real part of the said complex number characterizes the local absorptivity of the object, and the imaginary party of said complex number characterizes the local refractive index of the object. The overall phase is chosen appropriately when the origin of the three-dimensional frequency representation has a real value.
According to a variant of the invention, and in the case in which the origin of the three-dimensional frequency representation is one of the points that have been acquired, the microscope has means for dividing, by its value at the origin, the three-dimensional frequency representation obtained by the method pointed out in 3.5.1. This makes it possible to obtain a spatial representation in which the real part and the imaginary part of the complex numbers represent respectively the local absorptivity and the local refractive index. This also allows the entire representation to be normalized.
When the origin of the three-dimensional frequency representation is not part of the points that have been acquired, this operation is impossible. The operator who views an image sees, for example, the real part of the complex number, and must intuitively choose the total phase of the representation so as to obtain the most contrasted possible image.
3.5.3. Phase Correction in Relation to the illuminating Wave
The general phase correction algorithms defined above exhibit the drawback of being relatively complex to implement. A simplified version can be obtained when the acquisition system allows the acquisition of the non-diffracted part of the illuminating wave. This corresponds to the origin of the three-dimensional frequency representation of the object. This point is common to all the two-dimensional frequency representations.
The phase correction described in 3.5.1. can then be carried out in relation to the sub-representation part constituted by this single point. This correction can be grouped with the absolute correction described in 3.5.2. The total of the two corrections then is equivalent to dividing the entire sub-representation of the object obtained from a given illuminating wave by its value at the origin. When a plane frequency image is generated as an intermediate calculation step, this is equivalent to dividing the entire plane frequency image by its value at the point corresponding to the non-diffracted part of the illuminating wave. According to a variant of the invention, the phase correction of the two-dimensional frequency representations is carried out by dividing each sub-representation of the object by its value at the origin of the three-dimensional frequency representation of the object. This method is used, for example, in Embodiments 1, 2, 3 and 4.
3.5.4. Phase Correction in Relation to Pre-Recorded Phase Values
A three-dimensional frequency representation of the object is obtained from a series of two-dimensional frequency representations each corresponding to a different illuminating beam.
Each of these two-dimensional frequency representations can be phase corrected in relation to the sub-representation constituted by the origin alone, as indicated in 3.5.4. However, the high intensity of the corresponding point on each two-dimensional frequency representation makes difficult the simultaneous acquisition of the rest of the representation. According to a variant of the invention, the acquisition of plane frequency images takes place in two phases:
a preliminary phase during which are recorded the values obtained at the image point of the illuminating wave, for each illuminating wave;
an acquisition phase proper during which the direct beam can be obstructed and during which the values of the plane frequency images are recorded.
A two-dimensional frequency representation can then be obtained for each illuminating wave from these two recordings, the value at the image point of the illuminating wave being obtained from the first recording and the value at any other point being obtained from the second recording. The method described in 3.5.4. can then be applied to each two-dimensional frequency representation obtained.
When a series of two-dimensional frequency representations is obtained, for example to xe2x80x9cfilmxe2x80x9d the movement of cells, the preliminary phase must not be repeated. It only needs to be carried out once before the start of the acquisitions.
In order for this method to be functional, the phase difference between the illuminating beam and the reference beam, at the level of the receiving surface, must be reproducible. The illuminating beam generation systems used in Embodiments 3, 4 and 5 meet this condition. This phase correction phase is described for example in 7.18.1. and 9.18.2.
3.6. Vibration Compensation
The method described in 3.5.4. presupposes the reproducibility of the illuminating beams. The method described in 3.5.3., in the case in which several objectives are used, presupposes a constant phase shift between the waves received on each of these objectives. However, in Embodiments 3, 4 and 5, the vibrations can make these methods ineffective or less robust. In order for the results to be reliable, these vibrations must be compensated.
For this purpose, the system can periodically acquire a reference image. The reference image consists, for example, of an image obtained on the receiving surface for a fixed illuminating wave, which is not modified when the illuminating wave used to obtain the xe2x80x9cusefulxe2x80x9d plane frequency images varies. Each acquisition then corresponds to a reference image acquired at a near instant. A reference image acquired at an initial instant is chosen as absolute reference. By xe2x80x9cusefulxe2x80x9d image is meant an image obtained on the receiving surface and for which will be calculated a two-dimensional frequency representation used to generate the representation of the object.
With the vector v going through the entire support of an image, we denote by m(v) a xe2x80x9cusefulxe2x80x9d image obtained on this receiving surface, and h(v) the corresponding reference image. We denote as ho(v) the reference image chosen as absolute reference, obtained on the receiving surface. v represents the plane frequency projection and is thus a two-dimensional vector varying over the entire receiving surface. By "sgr"(v) is denoted the standard deviation of the Gaussian noise affecting the function h(v) at each point.
The phase variation of vibratory origin can be characterized, for example, by the coefficient   r  =                    ∑        f            ⁢              xe2x80x83            ⁢                                    h            0                    ⁢                      xe2x80x83                    ⁢                      (            v            )                    ⁢                      xe2x80x83                    ⁢                                    h              ⁡                              (                v                )                                      _                                                σ            2                    ⁢                      xe2x80x83                    ⁢                      (            v            )                                              ∑        f            ⁢                                    "LeftBracketingBar"                          h              ⁡                              (                v                )                                      "RightBracketingBar"                    2                                      σ            2                    ⁢                      xe2x80x83                    ⁢                      (            v            )                              
which represents the most probable phase difference between the reference images h(v) and ho(v).
The image m(v) can then be phase corrected as follows:
m(v)←r.m(v) where the sign←designates the sense.
When this preliminary correction has been carried out the images thus phase corrected can be used in the algorithms defined in 3.5.3. and 3.5.4.
If the system is totally free from vibrations, this preliminary correction is not required.
If the vibrations are of the low-frequency type, the reference image can be acquired only at a low frequency, however higher than the frequency of the vibrations of the system.
If the vibrations are of the high-frequency type, it is possible to acquire a reference image at each useful image.
In the presence of vibrations,
this preliminary correction is essential for the application of the algorithm defined in 3.5.4.
this preliminary correction is not essential for the application of the algorithm defined in 3.5.3. in the case in which a single objective is used. However, if several objectives are used, it allows the fixing of the phase difference between the plane frequency images generated from each objective. In this case, it is thus also essential.
This technique is used in Embodiments 3 and 4 and described in 7.17. It is also used in Embodiment 5 when the phase correction is carried out in accordance with paragraph 9.18.
A variant of the invention thus consists in periodically acquiring reference images corresponding to a fixed illuminating direction, and using these images to compensate for phase differences of vibratory origin affecting the plane frequency images.
3.7. Characterizing the Wave Vector of the Illuminating Wave
It is necessary to control the direction of the illuminating wave, i.e. its wave vector fe, for instance by mechanical means as described for example in [Turpin]. However, these mechanical means must be very accurate and are costly to implement.
In the conditions defined in 3.5.3., i.e. if the acquisition system allows the acquisition of the non-diffracted part of the illuminating wave, the non-diffracted part of the illuminating wave corresponds to the maximum modulus value on the frequency representation of the wave coming from the object. According to a variant of the invention, the microscope has means for determining the coordinates of this maximum and for calculating, from these coordinates, the wave vector fe of the illuminating wave. This method is used, for example, in Embodiments 1 and 2.
However, the presence of the object can slightly distort the value of the wave vector thus obtained. According to a variant of the invention, the wave vectors fe of each illuminating wave are obtained in a preliminary phase in which the object is eliminated or replaced by a transparent plate. The wave vectors thus obtained are thus not distorted by the presence of the object. This method presupposes that the wave vectors are reproducible from one acquisition to the other. On the other hand, it obviates the need to calculate these wave vectors according to mechanical parameters. This method is used in Embodiments 3, 4 and 5.
3.8. Characteristics of the Receiver
3.8.1. Use of a Microscope Objective
According to a variant of the invention, the receiver comprises a large-aperture microscope objective that transforms the rays coming from the object under a large aperture into paraxial rays that can be directed towards a receiving surface. This configuration offers better performance than the configurations defined in [Fercher] (absence of objective) or [Turpin] (small-aperture objective). While [Vishnyakov] uses such a configuration without benefiting from it, owing to the use of poorly suited tomographic methods, the algorithms defined in the present invention make it possible to take full advantage of this configuration.
3.8.2. Use of a Receiving Surface in a Frequency Plane
It is advantageous to directly acquire the image in the frequency domain, as in [Turpin 3]. This can be done, for example, by means of the receiver described in [Lauer 1] that allows improved performance compared to the receiver of [Turpin 3].
3.8.3. Use of a Receiving Surface in a Spatial Plane
Besides the microscope objective itself, the receiving system defined in 3.8.2. presents a paraxial part allowing the modification of the optical signal captured by the objective, to obtain a frequency representation. The signal from the objective first passes through the plane in which the objective normally forms the image of the observed sample. This plane will be called the spatial plane. It is then transformed by a paraxial system so that, in the plane in which the receiving surface is placed, a plane wave coming from the object has a point image. This plane, in which the receiving surface is placed, will be called the frequency plane. The paraxial part of the optical system used can include intermediate spatial or frequency planes. The receiving surface can be placed in a spatial plane, in a frequency plane, or in an intermediate plane. However, to simplify calculations, it will always be placed either in a spatial plane or in a frequency plane. In order for the received image to be correct, the following conditions must moreover be complied with:
if the receiving surface is in a frequency plane, the reference wave must be centered virtually at a central point of the observed object.
if the receiving surface is in a spatial plane, the reference wave must be the image of a virtual wave which is plane on the crossing of the observed object.
Under these conditions, the signal detected on a receiving surface placed in a frequency plane is the optical Fourier transform of the signal that would be detected in a spatial plane. One variant of the invention constituting an alternative to the receiver defined in 3.8.2. is thus to use a receiving surface positioned in a spatial plane and a reference wave which is the image of a virtual wave which is plane on the crossing of the observed object. A numerical Fourier transform then replaces the optical Fourier transform.
Embodiments 1, 3 and 4 use sensor means in a frequency plane and Embodiments 2 and 5 use sensor means a spatial plane. A plane frequency image can thus be obtained either directly on a receiving surface placed in a frequency plane, or by the Fourier transform of an image received on a receiving surface placed in a spatial plane.
3.9. Beam Attenuation
If the sensor is placed in a spatial plane, the direct illuminating wave has, as a representation on the sensor, a constant modulus value which is superimposed on the wave diffracted by the object. An excessively weak diffracted wave in relation to this constant basic level cannot be detected correctly. On the other hand, this basic level is not very high because the intensity of the reference beam is spread over the entire sensor, thus generally allowing good images to be obtained. If the sensor is placed in a frequency plane, the illuminating wave is concentrated at a point and, as in the previous case, an excessively weak diffracted wave in relation to this constant basic level cannot be correctly detected. As the wave is concentrated at a point, this basic level is high and this limitation is troublesome.
According to an advantageous variant of the invention, a device controlling the attenuation of the beam, having one or more attenuation levels, is introduced to solve this problem. The attenuation device makes it possible to obtain successively several recordings differing in the intensity of the illuminating wave. A less noisy value of the diffracted wave is then obtained by combining these recordings. The final value of the diffracted wave is calculated for example at each point from the recording for which the intensity of the wave received at the considered point is highest, but for which the sensor remains unsaturated at the considered point and in its immediate surroundings for all the interference patterns allowing said recording to be obtained.
Given the use of the beam attenuation device, the variant of the invention in which a plane wave has a point image allows the detection of weaker diffracted waves. In fact, the waves are not superimposed on the sensor with any other wave and very low levels can be detected when the illuminating beam intensity is high.
Such a device is used in Embodiments 1, 3 and 4.
3.10.
Illuminating Beam Generation System
Whatever the embodiment, an illuminating beam generation method must be designed. The methods proposed by [Turpin] have the drawback of requiring significant mechanical movements and hence of greatly slowing down the system.
An optical system, for example the one described in 8.1.1., can transform a parallel beam having a given spatial extension and variable direction into a parallel beam whose spatial extension has been reduced and whose directional variations have been amplified. In general, small directional variations applied to a beam having a broad spatial extension can be amplified by an optical system through a reduction in the spatial extension of the beam. As the spatial extension of the illuminating beam required for a microscope is small, this principle can be used for the optical amplification of small mechanical movements.
A beam with a variable direction can be transformed by a lens into a beam with a variable position in the back (image) focal plane of this lens. A directional variation of the beam in a part of its optical path is thus equivalent to a position variation in another part of its optical path and vice-versa. In intermediate planes, the variation is a joint position and directional variation. There is thus no reason to differentiate between a system generating illuminating wave position variations and a system generating directional variations, as these systems are equivalent.
According to one variant of the invention, the illuminating beam generation system comprises:
a beam deflector generating variations in a paraxial beam.
a large-aperture optical element (for example, a microscope objective or a condenser) transforming said incoming paraxial beam variations into significant directional variations in the outgoing beam.
Said system can also comprise a lens system designed so that the beam is parallel at the exit of said large-aperture optical element.
The beam deflector may, for example, be a mirror mounted on a positioner allowing the control of its orientation. This solution is implemented in Embodiments 1 and 2. However, this solution has two drawbacks:
the movement of the mirror generates vibrations which disturb the system. After each movement of the mirror, it is necessary to wait for the absorption of vibrations before proceeding with the acquisition.
the phase difference between the reference beam and the illuminating beam is not reproducible, thus preventing the use of certain algorithms such as those defined in 3.5.4.
Each of the beam deflectors described in 3.10.1., 3.10.2 and 3.10.3. enables these two problems to be solved.
3.10.1. Beam Deflector Based on a Series of Binary Deflectors
A system capable of sending the beam back in two directions can be built by means of a birefringent prism which transmits the ordinary beam and the extraordinary beam in two different directions. The laser beam used must then be polarized. A polarization rotator placed before the prism allows the orientation of its polarization in the ordinary direction or the extraordinary direction, which implies a different angle of deflection by the prism. However, available ferroelectric polarization rotators, which have the advantage of being fast, do not allow a rotation of 90 degrees but a rotation of about 80 degrees. This prevents the possibility of having both a beam polarized exactly in the ordinary direction, for one of the positions of the polarization rotator, and a beam polarized exactly in the extraordinary direction for the other position. Hence, in one of the positions, is created a spurious beam deflected in an undesired direction. To eliminate this spurious beam, it is necessary to use at the output of the birefringent prism a polarizer selecting only the desired beam. In order for this polarizer not to eliminate the beam in the other position of the rotator, a second rotator must be introduced between this polarizer and the prism, said second rotator being used to bring the electric field vector of the beam back to the passing direction of the polarizer when it is not there directly at the exit of the prism.
A system capable of returning a beam in many directions can be constituted by combining in series several of these elementary systems. By combining two of them, which produce a deflection of the same amplitude but in two different orthogonal directions, a doublet is formed. By combining N doublets in series, each doublet being characterized by birefringent prisms having characteristics such that the deflection angle of doublet number i is proportional to 2i, one obtains 2N possible deflection values in each direction. For example, with N=8, there is a total of 256xc3x97256 beam deflection directions.
According to a variant of the invention, the beam deflection system is constituted by the association of elementary deflectors in series, each of these elementary deflectors comprising a birefringent prism deflecting differently the ordinary beam and the extraordinary beam, preceded by an electronically controlled polarization rotator and allowing the orientation of the electric field vector of the beam along the ordinary axis or the extraordinary axis of said prism, and followed by a second rotator and by a polarizer allowing the elimination of spurious beams.
Such a device is used in Embodiment 3.
3.10.2. Beam deflector based on spatial modulators
A spatial modulator is a two-dimensional matrix of pixels allowing the phase or intensity modulation of a wave in a plane. Most spatial modulators are based on liquid crystals. Common LCD screens constitute an example of a spatial intensity modulator.
A spatial plane, on the path of the illuminating beam, will be defined by a plane in which this beam is parallel and is centered on the optical axis. A frequency plane will be defined as a plane in which this beam has a point image.
A parallel beam reaching a spatial plane has, in this plane, a complex representation of the form exp{j2xcfx80(fxx+fyy)} in which (x,y) are coordinates of a point of the plane and in which (fx,fy) are the coordinates of the projection of the wave vector in this plane. If a phase modulation device is placed in this plane and if a phase shift of the form xcex8=2xcfx80(gxx+gyy+c) is applied by means of this device, the wave has, after passing through said device, a complex representation exp{j2xcfx80((fx+gx)x+(fy+gy)y+c)}. The spatial modulation device has thus modified the direction of the incident wave. Wave vectors that can generate such a spatial modulation device are included in a cone the aperture of which depends on the maximum values of gx and gy permitted by the modulator. This cone will be called xe2x80x9cdeflection cone.xe2x80x9d
If an intensity modulation device is used instead of the phase modulation device, it is possible to apply an attenuation function of the type cos{2xcfx80(gxx+gyy+c)}. After passing through the device, the wave then has a form of the type exp{j2xcfx80((fx+gx)x+(fy+gy)y+c)}+exp{j2xcfx80((fxxe2x88x92gx)x+(fyxe2x88x92gy)yxe2x88x92c)} which corresponds to the superposition of two plane waves whose wave vectors are symmetrical in relation to the axis oriented along the wave vector of the wave leaving the device in the absence of modulation. One of the two waves can be stopped by a diaphragm, in which case the device constitutes a beam deflector comparable to the preceding one.
Intermediate modulation devices performing joint phase and intensity modulation may also be used.
A variant of the invention thus consists in using, as beam deflector, a suitably controlled spatial modulator.
According to a variant of the invention, said modulator is a phase modulator controlled so as to generate a phase shift in a form as near as possible to xcex8=2xcfx80(gxx+gyy).
Prior-art modulation devices operate pixel by pixel. This discretization leads to the generation of spurious frequencies outside of the deflection cone. One variant of the invention consists in eliminating these spurious frequencies by means of a diaphragm placed in a frequency plane on the path of the wave coming from the phase modulator.
Modulation devices allowing fast modulation are binary, i.e. to a given pixel there correspond only two possible phase or intensity values. The use of a binary modulation device results in the presence of a spurious plane wave symmetrical with the wave to be obtained in relation to an axis constituted by the direction of an undeflected beam. In the case of binary modulators, this is true even in the case of a phase modulator, whereas in the case of modulators generating a continuous modulation, this problem can be avoided by using a phase modulator. According to a variant of the invention, the diaphragm filtering the spurious frequencies is dimensioned so as to filter not only the frequencies located outside of the deflection cone, but also part of the frequencies located within the deflection cone, so as to stop the spurious plane wave.
Binary modulation devices also have the drawback of generating spurious frequencies included within the deflection cone and constituting a frequency xe2x80x9cnoise.xe2x80x9d According to a variant of the invention, these frequencies are stopped by an intensity modulator placed in a frequency plane along the path of the beam coming from the phase modulator, and controlled so as to allow only the sought frequency to pass.
Such a device is used in Embodiment 4.
3.10.3. Beam deflector consisting of a mobile mirror designed so that vibrations are not troublesome
The beam deflectors described in 3.10.1. and 3.10.2. are based on the use of liquid crystal devices and polarizers. These devices are not available in the ultraviolet radiation range. To use ultraviolet rays, other means are required.
In prior-art devices, the entire system was placed on an optical table.
According to one variant of the invention, the beam deflection device is made up of a mirror placed outside the optical table, the separation between the illuminating beam and the reference beam being provided by a separator fixed on the optical table and positioned after said mirror on the path of the beam.
As the mirror is placed outside the optical table, it does not generate vibrations in this table. As the separation of the beams takes place after the mirror, its vibrations also do not generate phase shifts between illuminating beam and reference beam. This consequently solves the problem of vibrations.
On the other hand, the fact that beam separation takes place after the mobile mirror means that the reference beam varies at the same time as the illuminating beam. This variation must be taken into account in the design of the system and compensated. For example, if the receiving surface is placed in a spatial plane, variations in the direction of the reference wave result in translations of the plane frequency image. According to a variant of the invention, this effect is compensated by providing a translation in the opposite direction of the plane frequency images obtained.
This technique is used, for example, in Embodiment 5.
3.11. Compensation of errors due to polarization
The image generation method described by [Wolf] is based on a scalar diffraction theory and assumes that the wave passing through the object is diffracted isotropically in all directions by each point of the object.
It is only on the basis of this assumption that the theoretical resolution of xcex/4 can be obtained. Scalar diffraction theory is however not valid for high diffraction angles. The intensity diffracted by a point of the object depends on the direction of the diffracted wave, the direction of the illuminating wave, the polarization direction of the illuminating wave and the polarization direction of the diffracted wave.
3.11.1. Compensation by real-coefficient multiplication
The wave diffracted by the object differs from the wave which would be diffracted if the diffraction were isotropic by a real multiplicative factor depending on the:
illuminating wave propagation direction
illuminating wave polarization
diffracted wave propagation direction
diffracted wave polarization
According to one variant of the invention, the microscope comprises means for determining this multiplicative factor and compensating it by multiplying the received waves by the inverse of said factor. Such a variant of the invention is described in 7.18.8.
3.11.2. Compensation in the case of an anisotropic material
If the observed object is made up essentially of an anisotropic material, the effects of the diffraction differ from what they are in an isotropic material. The material is then characterized at each point by six crystalline parameters plus absorptivity.
In the particular case in which the observed object is a uniaxial crystal, the refractive index of the ordinary ray has a constant value. According to one variant of the invention, a three-dimensional representation in which the complex numbers obtained characterize the absorptivity and the ordinary index of refraction can be calculated. According to a variant of the invention, this representation is calculated from plane frequency images obtained for an illuminating wave polarized so that it constitutes an ordinary beam.
As the ordinary polarization direction varies with the wave""s propagation direction, it is necessary to be able to modify the polarization direction of the illuminating wave. However, as the phenomena are linear, one need only record the wave received at every point for two polarization directions of the illuminating wave to be able to deduce the wave diffracted by the object for an illuminating wave of any direction. According to a variant of the invention, the microscope comprises means for generating two polarization directions of the illuminating wave. According to a further variant of the invention, the microscope also comprises means for analyzing the diffracted wave along two polarization directions, thus making it possible to distinguish the ordinary diffracted wave from the extraordinary diffracted wave. According to this variant of the invention, the plane frequency images are then obtained from four elementary images corresponding to each combination of two polarization directions and two analysis directions.
Such a variant of the invention is described in 7.18.9. In the description of 7.18.9. only the analysis direction corresponding to the ordinary ray is used, but it is also possible to take into account the analysis direction corresponding to the extraordinary ray. Two plane frequency images are then obtained for each illuminating wave propagation direction, one corresponding to the ordinary index and the other to the extraordinary index. The final frequency representation is obtained from this set of images, taking into account the variations in the extraordinary index at each point of the plane frequency images corresponding to the extraordinary index.
3.11.3. Compensation by combining several polarization and analysis directions
In the case of the isotropic material, the method described in 3.11.1. has the drawback of causing the noise level to rise considerably. One way to avoid this problem is to acquire at least four plane frequency images corresponding to each combination of two distinct polarizations of the illuminating wave and two distinct polarization directions of the diffracted wave. An appropriate algorithm then makes it possible, from these four images, to calculate a single image corresponding to the scalar parameter sought. According to a variant of the invention, the microscope consequently comprises means for generating two distinct illuminating wave polarizations, and means for analyzing the diffracted wave along two distinct polarization directions. A variant of the invention is that the microscope comprises means for calculating, from the plane frequency images corresponding to each combination of two polarization directions and two analysis directions, a single plane frequency image representing a complex scalar quantity complying with the condition of uniform diffraction in all directions. This principle is used in Embodiments 3, 4 and 5. The principle for calculating said scalar quantity is described in detail in 7.12.1.
In the case of wavelengths in the visible, said means for varying the illuminating wave polarization can consist of a liquid crystal polarization rotator. Said means for varying the diffracted wave analysis direction can consist of a polarization rotator associated with a polarizer. In the case of ultraviolet, these devices are not available.
3.11.4. Variation of illuminating wave polarization direction in the UV range
In the UV range, polarization rotators can be replaced by a quartz retardation plate rotating around an axis by mechanical means. However, these mechanical movements slow the system down considerably. For this reason, use is made of a system in which the only mechanical movements are those of shutters and in which the beam to be closed off has a spatial extension which is as small as possible so that the movement of the shutter is as small as possible. It is then possible to use a high-speed shutter or a shutter system with a rotating crownwheel.
According to one variant of the invention, such a system includes:
a beam separator which breaks the beam down into a beam A and a beam B.
lenses placed on each beam A and B and focussing these beams on points of focus at which the shutters are placed.
a device making it possible to again superimpose beams A and B having passed through their respective shutters.
a device placed on the place of one of the beams A or B, in the part of the path in which the two beams are distinct, and modifying the polarization of this beam.
Such a system can also comprise additional lenses designed to reform parallel beams after passage through the shutters. It can also comprise a second polarization modification device. The beam polarization modification device can be a retardation plate. The beam separation and beam superposition devices can be semi-transparent mirrors. Such a device is used in Embodiment 5.
3.11.5. Variation of analysis direction in the UV range
The wave coming from the object can be broken down into a wave whose electric field vector is parallel to that of the reference wave and a wave whose electric field vector is orthogonal to that of the reference wave. The intensity received on the receiving surface is the sum of the intensity of the wave whose electric field is orthogonal to the reference wave and the intensity produced by the interference of the reference wave and the wave whose electric field is parallel to the reference wave. The first of these intensities does not depend on the phase of the reference wave and consequently does not modify the complex value of the wave coming from the object measured by the combination of interference patterns corresponding to different phases of the reference wave. It is thus only the wave whose electric field vector is parallel to that of the reference wave which is obtained on the receiving surface.
The analysis direction of a wave can consequently be modified simply by modifying the polarization direction of the reference wave or, symmetrically, by modifying the polarization direction of the wave coming from the object.
According to a variant of the invention, the analysis direction is modified by varying the polarization direction of the reference wave or of the wave coming from the object.
According to a variant of the invention, the polarization of the reference wave or of the wave coming from the object is modified by a device comprising:
a beam separator breaking the beam down into two beams A and B
a retardation plate LA placed on the path of beam A and a retardation plate LB placed on the path of beam B, the angle between the neutral axes of these two retardation plates being 45 degrees.
The polarization direction of the beam having passed through retardation plate LA is then deduced from the polarization direction of the beam having passed through retardation plate LB by a rotation whose angle is the angle between the neutral axes of retardation plates LA and LB. If this angle is 45 degrees, the beams coming from plates LA and LB will always have orthogonal polarizations whatever the polarization direction of the incident beam.
The two beams A and B can then be combined by a superposition device after having passed through shutters, as in the case of the device described in 3.11.4. This has the drawback of requiring the use of shutters at a point of the beam path in which the direction of the beam is variable and in which the beam can consequently not be focussed on a fixed point.
According to a variant of the invention, the two beams A and B are separately superimposed on the beam coming from the object (if they constitute the reference wave) or the reference wave (if they come from the object). The interference patterns corresponding to each polarization direction are then formed on two distinct receiving surfaces.
Whether the plane frequency images corresponding to each polarization are obtained on distinct receiving surfaces or on the same receiving surface, a phase difference is then created between these images and must be compensated. According to a variant of the invention, the retardation plates are positioned so that the reference and illuminating beams reaching the receiving surface have different polarization directions, preferably 45 degrees from each other. Then, the part of the wave coming from the object which comprises frequencies near the illuminating wave is detected on the two receiving surfaces and can be used to calculate said phase difference.
3.12. System for eliminating direct illumination
The illuminating wave is generally much more intense than the diffracted wave. It can saturate the sensors or reduce considerably the signal-to-noise ratio of the system by requiring acquisition of high signal levels. The illumination of the non-diffracted part of the illuminating wave during the acquisition phase or during part of the acquisition phase clearly improves the performance of the system. In a frequency plane, the non-diffracted part of the illuminating wave has a point image and can be eliminated by placing an absorbing element on this point. According to one variant of the invention, the system accordingly comprises a device for eliminating the non-diffracted part of the illuminating wave, placed in a frequency plane, and absorbing the beam on a small zone around the point corresponding to this illuminating wave.
According to a variant of the invention, this device is made up of a spatial intensity modulator controlled to allow passage at all points except on a limited zone around the impact point of the non-diffracted part of the illuminating wave. This variant is implemented in Embodiment 4.
According to a further variant of the invention, this device comprises a glass which is mobile in translation within a frequency plane, an absorbing black spot placed on the glass being designed to stop the direct beam, the position of the glass being controlled so that this black spot coincides with the point of impact of the non-diffracted part of the illuminating wave. This variant of the invention is implemented in Embodiment 5.
3.13. Spatial modulator utilization device
The spatial modulators used in 3.10.2. or in 3.12. may in particular be high-speed binary modulators operating by reflection. They are generally used by means of a polarizing semi-transparent mirror which is a device of cubic form sending back incident beams in two different directions depending on their polarization. This device, owing to its thickness, generates a slight aberration which widens the point corresponding to a given frequency, which is detrimental to the quality of the images obtained.
To avoid the use of this device, it is possible to use beams coming from the modulator under an oblique angle. The incident and reflected beams are then separated. However, this method deforms the distribution of the generated frequencies. To prevent this deformation, said oblique angle must be small.
According to one variant of the invention, this problem is solved by using a device consisting of a mirror with two orthogonal reflecting faces and a lens traversed in one direction by the beam directed towards the modulator and in the other direction by the beam reflected by the modulator. The incident beam is reflected by one side of the mirror, passes through the lens, is reflected by the modulator, passes again through the lens in the opposite direction and is reflected by the second side of the mirror, resuming its initial direction. The incident and reflected beams can be partially superimposed on the lens, but are separated on the two-sided mirror. In order to obtain this separation on the mirror, while maintaining as small an oblique angle as possible, the two-sided mirror must be positioned approximately in one focal plane of the lens and the modulator must be positioned in the other focal plane of the lens.
3.14. Use of objectives traversed in both directions and/or of several objectives
The object can be illuminated on one side and the wave coming from the object can be picked up on the opposite side by an objective, thus allowing the reconstitution of part of the frequency representation of the object. However, other parts of the frequency representation of the object can be constituted only from the wave moving towards the side of the object from which the illuminating waves come. According to one variant of the invention, an objective is associated with an optical system allowing, on the one hand, the measurement on a sensor of the wave coming from the sample and having gone through the objective and, on the other, the formation of an illuminating wave which, after crossing the objective, becomes in the sample a plane illuminating wave of variable direction. This objective is then crossed in one direction by the illuminating wave moving towards the object, and in the other direction by the diffracted wave coming from the object. It plays both the role of objective receiving the wave coming from the object and the role of high-aperture system transforming the small directional variations of the wave coming from the beam splitter into large directional variations of the wave in the object. This can be achieved for example by means of a semi-transparent mirror placed on the path of the beam coming from the object and superimposing the beam coming from the object and directed in a given direction on the illuminating beam moving in the opposite direction. This variant of the invention is implemented in Embodiments 3, 4 and 5 which comprise several objectives. One arrangement in which only one objective is used is described in 7.18.10.
Possible limitations having to do with the direction of the illuminating wave, characterized by its frequency vector, influence the performance of the system. Maximum precision is obtained when all the possible directions are used. Similarly, it is desirable to record the wave diffracted by the object in all directions. When a single microscope objective is used, its aperture limits the directions in which it is possible to record the wave diffracted by the object. According to an advantageous variant of the invention, several objectives focussed on the sample are used, which makes it possible to record the wave coming from the sample along more directions. The objectives then cover almost the entire space around the sample, and the illuminating waves must necessarily pass through these objects to reach the sample. According to one variant of the invention, each objective is associated with an optical system allowing, on the one hand, the measurement on a sensor of the wave coming from the sample and having passed through the objective and, on the other, the formation of an illuminating wave which, after passing through the objective, becomes a variable direction plane illuminating wave in the sample. The illuminating waves can thus be generated in all the directions covered by the aperture of the objectives, and similarly the waves coming from the object can be measured in all these directions. Considering a variant of the invention, the acquisition and calculation system takes into account all the waves measured on all the sensors for all the illuminations used and, from these data, generates the three-dimensional frequency representation of the object. Each pair (illuminating wave direction-direction of wave from object) corresponds to a point of the frequency representation of the object, and the frequency representation thus generated consequently comprises all the points obtainable from the illuminating waves and the diffracted waves respectively produced and received by all the objectives.
A large number of objectives can be used in order to receive all the waves coming from the sample, or in order to increase the working distance by using low-aperture objectives. However, most of the samples observed in practice are flat and can be placed conveniently between two cover glasses. By virtue of a variant of the invention constituting the best compromise between utilization difficulty and performance, use is made of two large-aperture microscope objectives positioned opposite each other, the flat sample being introduced between these two objectives. This solution is used in Embodiments 3, 4 and 5. In Embodiments 1 and 2, which exhibit lower performance but are easier to fabricate, a single microscope objective is used.
3.15. Generation of inverse beams
When two or more microscope objectives are used, a plane frequency image is generated from the wave received by each objective. Each point of a plane frequency image corresponds to a given wave vector of the diffracted wave. In order to calculate the three-dimensional frequency representation of the object, it is necessary to determine these wave vectors correctly, and this in a coordinate system common to the wave vectors received by each objective.
Knowing the K factor and the optical center, defined in [Lauer 1], allows the determination of the wave vectors corresponding to each point of a frequency image. However, the coordinate system used for two-dimensional representations RA (before translation of vector xe2x88x92fe) reconstituted from the plane frequency image obtained from a given objective is different from that used for the representations RB obtained from the facing objective. To establish a correspondence between these two coordinate systems it is necessary to determine the coordinates of certain points in the coordinate system used for RA as well as in the coordinate system used for RB.
Each point PA of the representation RA is the image of a wave vector fe of the illuminating wave which reaches this point in the absence of an object, and has the coordinates of this wave vector. To this vector there corresponds a wave vector xe2x88x92fe of opposite direction whose image is a point PB of representation RB. The coordinates of point PB in a coordinate system used for RA are the opposite of the coordinates of point PA in this coordinate system.
The correspondence between the two coordinate systems can thus be established if the coordinates of point PB are also determined in coordinate system RB. This can be done by generating, by optical means, a wave vector beam opposite to the illuminating beam and by determining the coordinates of the image point of this beam in the coordinate system used for RB. If this correspondence is established at a sufficient number of points, the relationship between the coordinate systems used for RB and RA can be easily determined and these representations can be modified to use a common coordinate system.
In like manner, it is possible to obtain, by optical means, a direct correspondence between coordinate systems RB and RA. This requires the adjustment of a certain number of optical elements. To perform this adjustment, it is possible to check continuously the correspondence between the coordinates of point PB obtained in each of the coordinate systems used, and this for a certain number of points PB (three points in principle).
In both cases, it is necessary to generate a beam having the same characteristics as the illuminating beam, but propagating in the opposite direction. In general, given a beam used in the system, the term xe2x80x9copposite indicator beamxe2x80x9d will designate a beam having the same characteristics but propagating in the opposite direction.
Considering one variant of the invention, the microscope consequently comprises, during the 35 adjustment phase, means for generating an indicator beam opposite to the illuminating wave. These means may be eliminated after the adjustment phase corresponding to the determination, by calculation means or optical means, of the correspondences between coordinate systems RB and RA.
According to a variant of the invention, the microscope also comprises means for generating, during an adjustment phase, an indicator beam opposite to the reference beam. This beam will also be used in certain adjustment phases. For example, if the receiving surface is in a frequency plane, the reference wave is centered virtually at a central point of the object. The indicator beam opposite to the reference wave makes it possible to adjust the position of the objectives so that these objectives are focussed on the same point.
According to a variant of the invention, when the receiving surface is a spatial plane, an additional beam centered on this spatial plane is also used during an adjustment phase, as is its opposite indicator beam. This beam makes it easy, for example, to adjust the position of the objectives in the absence of a reference wave centered on a point of the object.
In Embodiments 3, 4 and 5, each beam used has an opposite indicator beam, and the means of generating these opposite indicators are described as forming part of the microscope and are not eliminated after the adjustments have been completed: shutters are simply used to cut off these beams.
According to one variant of the invention, the device generating an opposite indicator beam from an original beam comprises:
a semi-transparent mirror separating the original beam into an unmodified beam and a secondary beam.
a lens focussing the secondary beam in a focussing plane.
a mirror placed at the point of focus, which reflects the opposite beam back to said lens.
The fact that the mirror is placed at the point of focus guarantees that the reflected beam has exactly the opposite direction of the incident beam. The reflected beam passes through the lens again in the opposite direction. The part of this beam which is then reflected again by the semi-transparent mirror has the same characteristics as the unmodified beam but is directed in the opposite direction.
3.16. Determination of differences in the position of objectives
From waves coming from the object and passing through a given objective, it is possible to generate a three-dimensional representation of the observed object. In spatial representation, this representation is relative to a given origin, that will be called characteristic point of the objective. In general, characteristic points of the objectives used do not coincide. Consequently the part of the frequency representation generated from an objective is translated in relation to that obtained from another objective. This translation results in a frequency modulation, the points in the frequency space obtained from a given objective consequently being affected by a variable phase shift corresponding to this modulation. Considering a particular variant of the invention, the microscope comprises means for compensating this translation and generating a representation of the object in which the phase shift affecting each point of the representation is constant. To be able to superimpose the representations obtained from each objective, and in accordance with one variant of the invention, the microscope comprises means for determining the coordinates of the characteristic points of each microscope objective, in a common coordinate system. It is then possible to translate appropriately each representation before superimposing them. This translation in the spatial domain is equivalent to a demodulation in the frequency domain, which can be carried out directly on the plane frequency images.
3.16.1. Determination of coordinates of characteristic points of each objective
According to a variant of the invention, this can be obtained by using the beam centered on a central point of the observed object and its opposite indicator beam. This beam is received on one sensor after having passed through the objectives, and its opposite indicator is received on another sensor. From the beam received on a sensor, the two-dimensional frequency representation of this beam can be obtained and the coordinates of its point of focus can be determined. The point of focus of the beam is the same as that of its opposite indicator beam. The difference between the coordinates of the point of focus of the beam obtained from one objective and those of its opposite indicator obtained from another objective is equal to the difference between the coordinates of the characteristic points of these objectives in a common coordinate system.
This method can be implemented with any number of objectives, provided that the configuration is such that no group of objectives is optically isolated, i.e. that a given objective group, if it does not include all the objectives used, can always be reached by a beam coming from an objective outside the group. For example, if six objectives are used, they must be grouped two by two: each objective must receive beams coming from the other two objectives.
This aspect of the invention is implemented in 7.9.1. and in 9.12.
3.16.2. Determination of displacements of each objective
It is generally necessary to move the objectives in order to introduce the observed object. This operation modifies the coordinates obtained and calls for the repetition of the preceding operation. However, the presence of the object disturbs the beam which passes through it and prevents any precise result from being obtained. According to one variant of the invention, this problem is solved by using parallel beams of variable direction. A parallel beam has a point plane frequency image and the value obtained at this point is affected little by the local irregularities of the observed sample.
The difference between the phase of such a beam received on a sensor before the movement of the objectives and the phase of the same beam after the movement of the objectives depends on the vector characterizing the displacement of the characteristic point of the objective receiving the beam in relation to the characteristic point of the objective from which the beam comes. From these phase differences established for a sufficient number of beams, it is possible by means of an appropriate algorithm to determine this displacement. According to this variant of the invention, the phase of a set of parallel beams reaching a given sensor is consequently measured a first time in the absence of the object and a second time in the presence of the object. From the phase differences and possibly from the intensity ratios thus measured, an appropriate algorithm can recalculate the displacement of the characteristic points of each objective. These phase and intensity differences can be characterized, for each parallel beam, by a complex value obtained as the quotient of the value obtained in the presence of the object by the value obtained in the absence of the object at a corresponding point of the plane frequency image. This value will be called the phase and intensity ratio on a given parallel beam.
Knowing the initial coordinates of the origin of each representation and its displacement, its current coordinates can be deduced and the difference in position can be compensated. This method can be implemented with any number of objectives, with the same provision as in 3.16.1.
The first measurement in the absence of the object is carried out, for example, in 7.9.2. and in 9.13. In these two cases, the correction related to the position values determined as indicated in 3.16.1. is described in the same paragraph as that measurement.
The second measurement in the presence of the object and the calculation of the displacements are carried out for example in 7.11. and in 9.15. In both cases, the absolute positions are calculated directly without intermediate displacement determination, the correction related to the position values having already been performed as indicated above. This determination is coupled with the calculation of the average index of the object whose principle is given in the next paragraph.
3.17. Determination of index and thickness of the object
3.17.1. Principle
When two microscope objectives opposite to each other are used and when the observed object forms a layer between two flat cover glasses, the average index of the object, if it differs from the nominal index of the objective (index of optical liquid designed to be used with the objective), creates a spherical aberration which distorts the three-dimensional representations obtained. This spherical aberration is reflected in phase and intensity variations of the beams measured in 3.16.2., these variations depending on the index and thickness of the object. According to one variant of the invention, a program uses the values measured in 3.16.2. to determine simultaneously the displacements of the objectives, and the index and thickness of the object.
The calculation carried out in 7.11. or in 9.15. is an embodiment of this variant of the invention.
3.17.2. Minimization algorithm
For given values of displacements and of the index and thickness of the object it is possible to calculate the phase and intensity ratios on each parallel beam. According to a particular variant of the invention, the calculation algorithm determines the values of the displacements and the index and thickness of the object which minimize the standard deviation between the theoretical values thus calculated and the values actually measured.
This algorithm must determine an absolute minimum on a set of five parameters of a noisy function, the standard deviation function exhibiting, even in the absence of noise, local minimum values other than the absolute maximum. This problem consequently lends itself poorly to conventional minimization methods.
When the values of the parameters are known approximately, the algorithm can maximize, in a first phase, the value at the origin obtained by compensating the phase differences due to these parameters. Maximization of a function being equivalent to a minimization of its opposite, the term maximization will be used only to define the algorithm, but it would also be possible to speak of minimization.
According to a variant of the invention, the algorithm comprises iterative phases during which it determines, for each iteration, an absolute maximum on a set of vales of the parameters varying discretely, the function to be maximized having previously been filtered to eliminate the frequencies that would cause aliasing when sampling at the variation intervals of the parameters. According to this variant of the invention, the interval is reduced with each iteration and the central point of the set on which the parameters vary during an iteration is the maximum determined in the preceding iteration.
Such an algorithm makes it possible in general to converge towards the solution despite the local maxima and the large number of parameters.
Such an algorithm is described in 7.8.2.
3.18. Object position determination
The exact position of the object has no influence on the values measured in 3.16.2. and can consequently not be obtained from these values. On the other hand, it can modify the three-dimensional representations obtained: the spherical aberration affecting a given three-dimensional representation depends on the position of the object. If the index of the object differs from the nominal index of the objectives, this position must be determined.
The position of the object in relation to the objectives affects the proper superposition of the phase corrected two-dimensional representations. If it is not correctly evaluated and taken into account as a correction factor, abnormal phase differences appear between pairs of phase corrective two-dimensional representations, on the part of the frequency space that corresponds to the intersection of these representations.
According to a variant of the invention, the measurement of the position of the object in relation to the objectives comprises an acquisition phase during which a series of plane frequency images are acquired, corresponding to a series of illuminating beams of variable orientation. Knowing the position parameters, the index and the thickness of the object, previously calculated by the maximizing algorithm described in 3.17., and knowing the sought position, a program can determine the two-dimensional representations corresponding to each plane frequency image. According to this variant of the invention, the program for determining the position of the object in relation to the objectives is consequently a minimization program that determines the value of the position parameter that minimizes the abnormal phase differences. Such a program is described in detail in paragraph 17.15.
3.19. Use of objectives exhibiting aberrations
The design of an objective without aberrations is difficult. Aberrations increase in proportion to the size of the optical elements used, so that it is difficult to obtain a great working distance. It is also difficult to obtain a high numerical aperture.
In the present microscope, the complex value of the wave received on a receiving surface is recorded. The frequency representation of part of the wave coming from the object can be reconstituted when this part of the wave coming from the object reaches the receiving surface. The frequency representation of the wave coming from the object is obtained, in every case, by a linear relationship from the wave reaching the receiving surface. The only xe2x80x9cindispensablexe2x80x9d property of the objective is consequently its ability to pick up a significant part of the wave coming from the object, and transform it into a paraxial beam reaching the receiving surface. An objective having this property can easily be designed and have a high numerical aperture and long working distance. According to one variant of the invention, an objective affected by aberrations greater than the limit imposed by the diffraction is used, and the calculation program reverses the linear relationship between the wave coming from the object and the wave picked up on the receiving surface so as to compensate these aberrations.
According to a variant of the invention, the microscope objective is designed to comply with the following property:
(1)xe2x80x94The aberration affecting the image formed in the image plane of the objective must be less than a fraction of the diameter of the observed part of this image.
The requirement (1) ensures that, for the major part of the object studied, the entire beam coming from a given point reaches the receiving surface. In fact, the spatial extent of the observed sample is limited, depending on the systems, by a diaphragm or by the size of the sensor used. In the presence of spherical aberration, when the distance between a point and the limit of the observed area is less than the characteristic distance of the spherical aberration, part of the waves coming from this point do not reach the receiving surface and the image of this point can consequently not be reconstituted with precision. If the area concerned remains small in size, this drawback is not very troublesome: this is what is guaranteed by compliance with condition (1). If the area concerned were too large, the precision would be affected on the entire image.
Requirement (1) is similar to the usual requirement concerning spherical aberration, but is considerably reduced. In fact, results of good quality can be obtained with a spherical aberration of the order of 10 or so wavelengths, whereas in a conventional microscope spherical aberration must be a fraction of a wavelength.
However, an objective not having certain additional properties may be difficult to use. In fact, the linear relation linking the wave received on the receiving surface to the frequency representation of the wave coming from the object may be relatively complex. The algorithmic compensation of aberrations may thus require high computation volumes. According to one variant of the invention, this problem is solved by using an objective having, in addition, the following property:
(2)xe2x80x94The image, in the back (image) focal plane of a beam which is parallel in the observed object, must be a point image.
The plane frequency image used in all the embodiments is equivalent to the image formed directly in the rear focal plane. The requirement (2) means that each point of a plane frequency image corresponds to a given frequency of the wave in the object. If requirement (2) were not complied with, it would be necessary to use a generalized algorithm consisting in obtaining the value associated with a given point as a linear combination of the values detected on a set of neighboring points. Requirement (2) is relatively simple to comply with. The use of an objective not complying with condition (2) would only be of limited interest, but would considerably complicate the calculations needed to obtain an image. An example of the use of an objective complying with conditions (1) and (2) is described in paragraph 7.21.
In the case of an objective complying only with conditions (1) and (2), the image reconstitution algorithm must take into account the relation between the coordinates of a point on the plane frequency image and the corresponding frequency of the wave in the object.
By B(xcex1) is denoted the image point, in the back focal plane, of a parallel beam in the object and forming in the object an angle a with the optical axis.
According to a variant of the invention, the algorithms are simplified by using an objective complying in addition with the following condition:
(3)xe2x80x94The distance between point B(xcex1) and point B(0) must be proportional to sin xcex1.
Compliance with condition (3) means that the coordinates of a point on the plane frequency image must be directly proportional to the components along two corresponding axes of the wave vector of the wave in the object, which considerably simplifies the calculations. If an objective complies with conditions (1)(2)(3), spherical aberration results only in a phase shift of each point of a given frequency representation, a phase shift which can then be easily compensated. An example of the use of an objective complying with conditions (1)(2)(3) is described in 7.20. Condition (3) is roughly equivalent to the condition of Abbe""s sines.
In an objective complying only with conditions (1)(2)(3), the image is disturbed in the vicinity of the diaphragm over a distance equivalent to the characteristic distance of the spherical aberration. This disturbance is eliminated by the use of a conventional objective rid of any spherical aberration, which thus makes it possible in principle to obtain the best images. However, the use of an objective complying only with (1)(2)(3) allows a significant reduction in the design requirements of the objective. This makes it possible to obtain an objective with a longer working distance, a less expensive objective, or an objective with a higher numerical aperture. This technical solution may thus be preferable in certain cases.
The use of such a microscope objective requires the algorithmic compensation of phase differences affecting the plane frequency image and which are the consequence of spherical aberration. This calls for the determination of the function characterizing the spherical aberration induced by the microscope. Because the objective exhibits a symmetry of revolution, the phase difference affecting a point of a plane frequency image depends only on the distance between this point and the optical center. The spherical aberration may thus be characterized by a one-dimensional function representing the phase difference as a function of this distance.
According to a variant of the invention, this function can be obtained by the optical calculation program used for the design of the objective. In fact, this program allows the tracing of beams and can easily be improved to also enable optical path calculations. As the phase differences are proportional to the optical path differences affecting the rays coming from a given point, they can be deduced from geometrical considerations of this type.
According to a further variant of the invention, this function can be measured by optical means. In accordance with this variant of the invention:
two identical microscope objectives facing each other are used.
use is made of an illuminating wave centered on a point of the plane at which a first objective normally forms the image of the observed sample.
the wave received in the plane at which the second objective forms the observed sample""s image is detected.
In the absence of spherical aberration, the phase of the detected wave must be constant. In the presence of spherical aberration, the phase difference due to the aberration is twice the difference due to a single microscope objective. This makes it possible to obtain the sought function.
3.20. Compensation for spherical aberration and position differences
Spherical aberration due to the refractive index of the object, spherical aberration due to the properties of the objective, and positioning errors relative to the objectives, all result in phase differences applied to the plane frequency image. These phase differences must be corrected in order to obtain a good quality image.
According to a particular variant of the invention, this corrected is carried out by multiplying each plane frequency image by a correction function taking into account the various parameters determined above. The calculation of such a function is described in 7.16. and the multiplication operation is carried out for example in step 2 of the algorithm described in 7.17. If an objective exhibiting spherical aberration is used, the calculation described in 7.16. must be modified as indicated in 7.20.
3.21. Regular sampling
When two microscope objectives are used, the sampling interval on the plane frequency image generated from one of the receiving surfaces can be taken as the basis for the sampling interval of the three-dimensional representation of the object along two corresponding axes. If no precaution is taken:
The image points of the illuminating waves on this plane frequency image do not correspond to integer values of the coordinates in pixels.
If two objectives are used, the sampling interval and the axes on the plane frequency image generated from a receiving surface associated with the objective located opposite do not correspond to the sampling interval and to the axes of the three-dimensional representation of the object.
The result of this is that the sampling of the three-dimensional representation of the object is not regular. According to one variant of the invention, this sampling is made regular along two axes corresponding to the axes of the plane frequency representations. The quality of the three-dimensional representation of the object is then clearly improved.
The plane frequency image can be modified in particular, due to imperfections in the optical system, by rotation or by similitude transformation. To obtain a regular sampling, it is necessary to cancel or compensate for these imperfections.
Further, in Embodiment 4, there must be a point to point correspondence between the different SLMs used and the CCDs. To obtain these correspondences, adjustments of the same type are needed.
According to a variant of the invention, the microscope consequently comprises one or more optical devices allowing a rotation adjustment of the images generated in the frequency planes and/or one or more devices allowing adjustment by magnifying the images generated in the frequency planes.
3.21.1. Adjustment of representation scale (homothetic transformation)
This adjustment is in fact a magnification adjustment. According to a variant of the invention, the magnification of an image is adjusted by means of an optical system with a variable focal length. Such a system may, for example, be composed of two lenses, a variation in the distance between said lens resulting in a variation in the overall focal length. Such a device is used in Embodiments 4 and 5 and is described in 8.1.4.1.
3.21.2. Image rotation adjustment
According to a variant of the invention, said adjustment is carried out by means of a device consisting of a first group of fixed mirrors and a second group of mirrors, complying with the following conditions:
the first group of mirrors symmetrizes the wave vector of the incident beam in relation to a given axis.
the second group of mirrors symmetrizes the wave vector of the incident beam in relation to a second axis.
the second group of mirrors is mobile in rotation around an axis orthogonal to the plane of these two axes.
Both groups of mirrors then have the effect of imparting a rotation to the beam represented in a frequency plane, the angle of rotation being twice the angle between the two axes of symmetry. Such a device is used in Embodiments 4 and 5 and is described in 8.1.4.2.
3.22. Phase shift system
The phase shift system used can be a piezoelectric mirror, constituting the most usual solution. However, used at high speed, such a mirror generates vibrations. According to an advantageous variant of the invention, the phase shift system used is a birefringent blade inducing a phase shift of 120 degrees between its neutral axes, preceded by a polarization rotator allowing the orientation of the electric field vector of the beam along one or the other of said neutral axes, and followed by a second polarization rotator for bringing the polarization direction of the beam at the output of the device back to its direction at the input of the device. As this system allows only a phase shift of 120 degrees, it is necessary to combine two in series to obtain a phase shift of xe2x88x92120, 0, or +120 degrees.
3.23. Data processine method in the case of a limited RAM
Three-dimensional frequency representation calculations involve large quantities of data. As these data are normally accessed in a random order, they cannot be stored during calculations on a sequential-access medium such as a hard disk and must be stored on a random-access medium such as an internal computer memory (RAM).
According to an advantageous variant of the invention, adapted to the case in which the system does not have sufficient RAM to allow storage of all the data, the calculation algorithm is modified so as to process the data block by block, a block corresponding to a large amount of data which can then be stored sequentially on a sequential-access medium and loaded into central memory only during the processing time of said block. For this purpose:
The modified algorithm carries out, within a three-dimensional space, processing operations horizontal plane by horizontal plane, each horizontal plane being stored on the sequential-access medium in a single block.
In order to be able to also carry out processing along the vertical dimension, the algorithm incorporates axis exchange phases which make it possible to bring the vertical axis temporarily into a horizontal plane.
The axis exchange procedure works block by block, the blocks generally having equal or similar dimensions along the two axes to be exchanged and having as byte size the maximum size that can be stored in the central memory of the system (random-access memoryxe2x80x94RAM).
This method is implemented in Embodiment 1 and described in paragraph 5.21.
3.24. Images venerated by the microscope
The three-dimensional representations generated by the present microscope can be stored and transmitted in the form of a three-dimensional array of complex numbers. According to one variant of the invention, it is possible to generate two-dimensional projections or sections representing either the refractive index or absorptivity in the object.
In the case of a projection, one generates a projection of the three-dimensional image on a projection plane and along a projection direction orthogonal to the projection plane. Each of the projection plane is obtained from all the values of the three-dimensional spatial representation which is located on a line passing through this point and directed along the projection direction.
According to a variant of the invention, the value associated with each point of the projection plane is obtained by extracting the maximum value of the real or imaginary part or the module from the points of the three-dimensional spatial representation located on the corresponding line.
According to a variant of the invention, the value associated with each point of the projection plane is obtained by integrating the complex value of the points of the three-dimensional spatial representation located on the corresponding line. It is then possible to display either the real part or the imaginary part of the projection thus obtained. According to this variant of the invention, the projection can be obtained more rapidly as follows, in two steps:
step 1: extraction, in frequency representation, of a plane passing through the origin and orthogonal to the projection direction.
step 2: inverse Fourier transformation of this plane.
The two-dimensional array thus obtained constitutes a projection along the direction having served to extract the frequency plane.
3.25. Optical element positioning system
The embodiments described require the use of many high-precision positioners. These positioners are costly elements ill suited to mass production and capable of losing their adjustment with time.
According to a variant of the invention, this problem is solved by using removable positioners during the manufacture of the microscope, each element being positioned and then fixed by a binder, for example an adhesive, and the positioner being removed after final solidification of the binder.
3.26. Shocks, vibration and dust protection system
The microscopes described consist of a set of elements fixed to an optical table. During possible transport, shocks, even minor ones, can lead to the misadjustment of the system. During prolonged use, dust may deposit on the different optical elements.
According to one variant of the invention, the greater part of the optical device is included in a hermetically closed box which is itself included in a larger box, the link between the two boxes being provided by shock absorbers placed on each side of said hermetically closed box. This system protects the microscope from shocks and from dust while providing a good suspension for the optical table.